Repository navigation
test: add a launch smoke test for the built payload - #3
Closed
tylerdotai wants to merge 2 commits into
Closed
tylerdotai wants to merge 2 commits into
tylerdotai wants to merge 2 commits into
Conversation
The offline suite asserts on files; nothing proved that a built payload actually starts. A package can build cleanly and still be broken at runtime -- a native addon that cannot be dlopen'd, a runtime resource that is not where the app looks for it, an addon built against the wrong Electron ABI -- and none of that is visible to a build that only asserts on files. tests/smoke-launch.sh launches the staged payload under Xvfb with an isolated HOME/XDG tree and waits for the app's own startup-readiness signal (outcome:"ready" with failed_count: 0), so the check is on the app's report rather than on a proxy signal. It fails if the app exits first, times out, reports a failed startup subsystem, or dies after reporting ready. build.yml runs it after "Build packages", with the xvfb package added and MMX_SMOKE_REQUIRE=1 so a missing Xvfb is an error in CI rather than a silent skip. Verified against a payload built by this repo's install.sh from the official macOS 3.1.4 x64 DMG: ready in ~3s, exit 0. Against a payload whose launcher exits immediately: exit 1.
The launcher is started with 'cd $PAYLOAD' and an absolute path, but $LAUNCHER was built from $PAYLOAD as given. With a relative payload directory (which is how the CI calls it: ./tests/smoke-launch.sh build/payload) the launcher resolved to build/payload/build/payload/start.sh and the app never started. Resolve $PAYLOAD to an absolute path first.
Author
|
Closing this in favour of a single consolidated issue (#6) — four findings in one place, so there is one thing to read and reply to rather than three overlapping PRs. The work is not lost: the branch is still up in the fork and I will open a PR for any part of it you want. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The offline suite in
tests/run.shasserts on files. Nothing in the repoproves that a built payload actually starts.
That gap is not academic. A package can build cleanly and still be broken at
runtime:
dlopen'd (wrong ELF class, missinglibcua_driver_sdk.so, an addon built against a different Electron ABI),None of that is visible to a build that only checks that files exist — and the
build already warns about this class of problem in its own comments:
Change
Add
tests/smoke-launch.sh, which launches the built payload headlessly andwaits for the app's own startup-readiness signal, plus a CI step that runs
it against the staged payload.
The app logs its lifecycle timings once every startup subsystem has
initialised.
outcome":"ready"withfailed_count: 0is that contract, sothe test asserts on the app's own report rather than on a proxy signal (a
window appearing, a process staying alive).
The script:
Xvfbdisplay and forces the Electron X11 backend, becauseElectron prefers Wayland and there is no compositor under Xvfb,
HOME/XDG_*tree, so the testneither touches a real profile nor collides with a running desktop session's
single-instance lock,
out, reports a failed startup subsystem, or dies after reporting ready,
session of the same app is never touched.
It is a separate script rather than a new case in
tests/run.sh, becausethat suite is explicitly "no network, no DMG, no GUI"; this one needs a
display and a real build.
Verification
Against a payload built by this repo's own
install.shfrom the officialmacOS 3.1.4 x64 DMG (Electron 42.8.0):
Against a payload whose launcher exits immediately:
So the test fails on a broken payload and passes on a good one, in both
directions.
CI
build.ymlgains thexvfbpackage and one step after "Build packages":MMX_SMOKE_REQUIRE=1makes a missing Xvfb a hard error in CI instead of askip, so the step cannot silently pass for the wrong reason. Locally the
script skips with a clear message when Xvfb is absent.
Notes
start.sh, or a bareElectron binary with
resources/beside it (the upstream app layout), soit can also be pointed at a payload built another way.
slow the build.